On the use of stochastic differential geometry for non-equilibrium thermodynamics 

modeling and control 
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We discuss the relevance of geometric concepts in the theory of stochastic differential equations for 
applications to the theory of non-equilibrium thermodynamics of small systems. In particular, we 
show how the Eells-Elworthy-Malliavin covariant construction of the Wiener process on a Riemann 
manifold provides a physically transparent formulation of optimal control problems of finite-time 
thermodynamic transitions. Based on this formulation, we turn to an evaluative discussion of recent 
results on optimal thermodynamic control and their interpretation. 

PACS numbers: 05.70.Ln Nonequilibrium and irreversible thermodynamics, 02.50.Ey Stochastic processes, 
02.50.Ga Markov processes, 02.30.Yy Control Theory 



INTRODUCTION 



Stochastic differential equations [19j provide a widely 
applied mathematical model for non-equilibrium dynam- 
ics. In particular, they are well adapted to the descrip- 
tion of kinetics and finite time thermodynamics of small 
systems such as bio-molecules, RNA and other molec- 
ular scale "machines", see e.g. 38] for review of the- 
oretical and experimental aspects. The application of 
stochastic differential equations to non-equilibrium phe- 
nomena bears a natural relation to geometry in an, at 
least, twofold way. First, even when a stochastic differ- 
ential equation is used to model processes evolving in a 
flat, Euclidean, space, in arbitrary coordinates the scale 
of the noise in the stochastic differential equation imposes 
a Ricmannian or sub-Ricmannian (if there are constraints 
on the admissible directions of motion such as condi- 
tioning a set of variables to be the derivatives of other 
ones) metric on the space. The second way geometry sets 
in is more distinctive of applications to non-equilibrium 
physics. It originates from the use of second derivatives of 
thermodynamic potentials to impose a Riemannian met- 
ric on the set of equilibrium states. The consequent no- 
tion of thermodynamic length has been then exploited to 
characterize optimal processes in macroscopic and, very 
recently, microscopic, nano-scale processes (see [431 ] for 
references and discussion). In the present contribution we 
show how a coordinate-independent geometric formalism 
may ease the analysis of control problems arising in the 
study of finite-time thermodynamics of small systems. 
Although the needed mathematical tools are well known 
in stochastic analysis (see e.g.0, [l6, lj|, 37|), we are not 
aware of any previous application to stochastic thermo- 
dynamics. In what follows we will generically denote by 
M a complete connected Riemannian manifold. We will 
also imply some further technical assum ptio ns to guar- 
antee probability conservation (see e.g. [17| for concise 
discussion). In practice, our main focus here will be on 
covariance of physical laws so that M can be thought as 
M. d endowed with a Ricmannian metric. Finally, in order 



to simplify the notation, thermodynamic expressions will 
be evaluated at unit temperature. 



GEOMETRY AND THE DEVELOPMENT MAP 

Differential geometry is most naturally formulated 
in the language of first order forms. For this reason 
Stratonovich calculus [l9| is commonly considered to 
have an edge on Ito calculus when it comes to iden- 
tify geometric structures. Let us therefore start by con- 
sidering a diffusion process £ = {| t ;iG [t ,*f]} speci- 
fied by stochastic differential equations in the sense of 
Stratonovich 



d& = M6,i)d* + e<(6)*dftit 



(1) 



in a coordinate neighborhood U G M centered e.g. 
around the initial condition £ to = x Q . The sym- 
bol pinpoints the Stratonovich prescription. In ([T]) 
65 : U x [t Q , if] 1 — ^ M. d is some smooth map, Einstein con- 
vention is implied on the latin indices i = l,...,d to 
pair the elements of a collection of d independent one- 
dimensional Euclidean Wiener processes (Brownian mo- 
tions) fa = {/3j ; t; t G [t , if]} to d orthonormal frame fields 
i e i}i=i specifying a basis for T^ t M. Let us also endow M 
with a strictly positive, time-independent metric tensor 
g and require that for any xell 

( e i i e j)g '-~ Ba 1 a 2 e i le j' 2 = (2) 

(otj = 1, . . . , d, j = 1, 2) which is equivalent to 

e l ® ei = g _1 (3) 

The hypothesis of time-independence of the metric tensor 
is physically relevant and will also serve later to ncaten 
the formalism. The extension to the time dependent case 
is, however, straightforward and it is of common use for 
example in the inquiry of Ricci flows, see e.g. [1, 13] 
for details. From (p} we can write for any x G U the 
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generator of the stochastic process x acting on any scalar 
function / :U4l 



1 



£s/ = ( b s ■ d x + -e ■ d x e ■ d x ) f 



with Alb the Laplace-Beltrami operator and b a vector 
field which we will refer to as the covariant drift. In 
general, it is not possible to identify globally on M the 
Stratonovich drift with the covariant one i.e. b$ = b 
unless the integrability condition 



b,s + -^(ei ■ d x ei) 



d x + \g~ l :d x ®d x jf (4) Vl . dx 



[v 2 ,e) g = v 2 ■ d x (v 1 ,e) g 



(9) 



( b d x = b ai d x ^ and A : B = TrA+B). The last equality 
in ^ motivates the identification of the inverse of the 
metric as the diffusion tensor of the process £. A dif- 
fusion process is fully specified by the knowledge of the 
conditional expectation of its increments up to second 
order. Under the present hypotheses these expectation 
values are 



lim E-t 

dtj.0 



dt 



1 



b s + -j(e-i ■ d x ei) ) (x,t) := bj(x,i) 



(5) 



the "Ito drift" , and 

(£t+dt - £t) ® (£t+dt - it) 



lim E x ( 

dtj.0 



dt 



= g~ 1 (*) (6) 



the diffusion tensor. It is therefore physically justified 
to take ([5]) and ^ as the data specifying a stochastic 
differential equation. We are now in the position to pin- 
point two disadvantages related to the use of ([lj. For 
any £ 0(d), the group of orthogonal matrices, the 
vector- valued Wiener processes (3 := [/3i, . . . , $4] an d 0/3 
are statistically equivalent. Let us denote with 0(M) 
the collection of d-tuples [x, e\(x), . . . , Bd(x)] attached 
to any x £ M. In the language of differential geom- 
etry, this collection forms the "bundle" of orthonormal 
frames specified by the triple (0(M), 0(d), M) [TJ, 
Since only © is observable, any element of the bundle 
must provide an equivalent description of the statistics 
of the process £. Equation (TTJ), however, is not invariant 
for different choices of orthonormal frames in the bundle. 
More explicitly, the same statistics can be equivalently 
described by the Stratonovich drift bs or by its "gauge" 
transform 



6s [0] 



f 



(7) 



This fact was already noted long ago in connection to the 
inquiry of the covariant path-integral representation of 
stochastic processes and Euclidean Quantum Mechanics 
over curved manifolds (see e.g. [13, and [33| for a 
more complete list of references). A second disadvantage 
is that the natural, coordinate independent, definition of 
the generator of £ acting on scalar functions is [2lL |22T | 



£ = b ■ a 



I . 
2 



(8) 



is satisfied for any constant vectors v\, v 2 . To interpret 
([9]) we observe that it is satisfied if there exists a collec- 
tion of scalar functions {Gi\ d i=Y such that 



d x G l = g ■ e t 
whence it follows that 

<e,,d&) g = dG 4 (& 



(10) 



(11) 



meaning that there exists a change of variables turning in 
(HI the frame fields {ei}^ =1 into the canonical basis of R d . 



In other words, (|T0|) states the existence of a priviledged 
choice of global coordinates for which the noise becomes 
additive. In general, however, ((9|) is not satisfied since 



A LB f = g- 1 : d x ® d x f - (V : g" 1 ) ■ d x f 



(12) 



with T the Christoffel symbols of the Levi-Civita con- 
nection on M. In 2l| Ito gave the expression in local 
coordinates of a stochastic differential equation (in Ito 
sense) associated to (JSJ 



d€t = 



b -ir 

2 



g 1 dt + e,dft ;t 



(13) 



A straightforward calculation shows then that (fl} is 
equivalent to ([IB"]) if © is satisfied. The conclusion 
is that the Stratonovich equation (fT|) does not provide 
a coordinate-independent description of the dynamics. 
Furthermore, inspection of (|13[) shows that the "Ito drift" 
cannot transform as vector field under general change 
of coordinates. The Eells-Elworthy-Malliavin develop- 
ment map (see e.g. [El [l(l EH for rigorous and pedagogic 
derivations) obviates these disadvantages. The idea is 
that a Wiener process can be path-wise constructed on 
M by "rolling" the manifold along the realizations of an 
Euclidean Brownian motion. Mathematically this means 
that the Wiener process should be constructed as the so- 
lution of the system of Stratonovich differential equations 



duj t = e^d/3 r 



de. 



T : e i:t (g) duj t 



(14a) 



(14b) 



As above, the {fti} i=1 are a collection of independent 
Wiener processes. Were they constant vectors, ([M]) could 
be couched in the form of a geodesic equation. Further- 
more, upon converting (|14a[) to the Ito representation we 
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recover (|13[) for vanishing covariant drift. An important 
further consequence of (I14a[) is that we can identify a 
stochastic process £ as a local M- valued martingale if in 
any local chart it satisfies 1|1 2t| 37 1 



d& + ~r 
d€* + ir 



g _1 di = local Euclidean martingale (15) 



Three remarks are in order before concluding this short 
discussion of background results from stochastic analy- 
sis. First, (Q~4|) admits a straightforward extension to 
semi-martingales, simply by introducing a drift term into 
(|14a[) . We do not need to take this step here explicitly, 
since we can combine Girsanov formula and the develop- 
ment map (fT4|) to take into account drift terms (l8j |: 



dP € 
~d~P~ 



A 



(b,e iit d/3 iit ) 6 -- 



2 I 



(Vb+||b|H)dt 



for 



V6 = 



3*- (v1i|6 Q ) 



(16) 



(17) 



The symbol V betokens here and in what follows the co- 
variant derivative operation compatible with the metric 
g. The stochastic integral in the first row of (fT6| is in Ito 
sense. The use of the time-symmetric Stratonovich inte- 
gral in the second row of (fT6"|) exhibits that the argument 
of the exponential transforms indeed as a scalar under a 
change of coordinates. Second, if the metric is time de- 
pendent we must include a drift term l/2<9 t g _1 in (|14b[) 
to preserve the metric compatibility of the connection. 
Finally ^ the adjoint of © 

1 



fit/ = -V-6/- 



-ALBf 



(18) 



is defined with respect to the invariant Riemann volume: 

d^x := d d x v / detg(a;) (19) 

The covariant density k is then related to the transition 
probability density with respect to the Lebesgue measure 
p by the equation 



p(x 2l t 2 \x 1 ,t 1 ) = y/ det g(x 2 ) k(x2,t 2 \xi,ti) (20) 

for any coordinates x„ i = 1,2 in a local chart U, and 
any t 2 > h in [t a ,tf]. 



KULLBACK-LEIBLER DIVERGENCE AND 
TIME-REVERSAL 

Our aim is now to derive with the help of the de- 
velopment map the covariant expression of time rever- 
sal relations which can be used as bridge relations be- 
tween the theory of stochastic processes and finite-time 



thermodynamics of small systems. Under rather general 
smoothness assumptions 35. [36|. if we know the prob- 
ability m = -^/[gjn and transition probability densities 
P = x/lslk. of a semi-martingale process £ in the full 
time horizon [if , t a ] , we can construct the time reversed 
time evolution by requiring for any t Q < t\ < t 2 < tf 

n(x 2 ,t 2 )k {r ' ) (x 1 ,t 1 \x 2 ,t 2 ) = k(x 2 ,t 2 \x 1 ,t 1 )n(x 1 ,t 1 l21) 

If £ is adapted to a M- valued Wiener process (fl"4|). we 
can use Girsanov formula (|16|) to evaluate averages of 
non-anticipative functionals of £ as averages with re- 
spect to w. As noticed in [3(|, we can also take advan- 
tage of the invariance under time reversal of the Wiener 
process to express the density of a backward process 

1= {It; t e [t , if]}: 



dFj 
dP, 



<b,e i;t >d/3 i;t } g 



llblljdt 



(U>i,t) 



Jt f { 



(db> t )g 



(22) 

The symbol > appearing in the first row of (|2"21 indicates 
that the stochastic integral is defined by taking the limit 
of Riemann sums sampled according to the post-point 
discretization; in other words, the first stochastic integral 
in (|22l) is an Ito integral with respect to the filtration of 
the "future" while the second is a Stratonovich integral. 



law 



The requirement £ = £ translates into 



U ( UJ t 2 ,t 2 )— 5 -{uj) 



x dP f , s 
n(w tl ,ii)— ^-(w) 



(23) 



dP^ v ' v 11 y dP 

whence we immediately recover the classical result 

b(x,t) ^b{x,t)+g- 1 -d x ii(x,t) (24) 

Both sides of the equation are now well-defined vector 
fields. We thus see that the same probability measure 
admits a forward and backward dynamics representation 
in terms of two different drift vector fields. A natural way 
to compare the two vector fields with respect to the same 
filtration is to introduce a forward, auxiliary process £ 
absolutely continuous with respect to £ defined by the 
replacement 



b^b= b 



(25) 



in the stochastic differential equation driving The ra- 
tionale behind (|2"5|) is to combine the stochastic with 
deterministic time reversal. As a consequence, for a 
gradient -type drift (see equation (f33|) below) at equilib- 
rium (|25|) reduces to the identity 6 = 5. In general a 
natural quantifier of the "drift discrepancy" is obtained 
by considering the Kullback-Leibler divergence [23| of the 
process £ with respect to £: 

3C(P f ||P e ) :=E«)ln^«) 



E<«) dP « 



dPc 



(u>)ln 



dP c dP^ 
dP^dPl 



0*0 (26) 
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The notation E^) ( E^) ) emphasizes that the average 
is with respect to the measure of £ (u>). The Kullback- 
Leibler divergence is by construction a positive definite 
quantity. Direct evaluation of (f2l)f yields 



aC(P € -||Pc)=E(«)^ t '(6 + 6td^) g 
_ E tt) f\ ^-^ + b)+\\b\\l 



(27) 



Upon inserting (fMf into (f2~Tf and using probability con- 
servation and the covariant Fokker-Planck equation 



(d t - £ f )n = 



(28) 



we can prove that the Riemann integral in (|27[) vanishes. 
The integrand of the Stratonovich stochastic integral is 
instead proportional to the current velocity 3^, 3(| of the 
process £: 



b+b 



v — 



(29) 



The current velocity enjoys two important properties. 
First, it plays for expectation values of Stratonovich line 
integrals the same role as the Ito drift ([5]) for expectation 
values of Ito line integrals: for any smooth, mean square 
intcgrable vector field h the equality 



E«) j\h°d£) g = E^ J\\t (h,v) t 



(30) 



holds true under the assumption that integrations by 
parts do not bring about boundary terms. Second, 
in terms of the current velocity the covariant Fokker- 
Planck equation reduces to deterministic mass conserva- 
tion equation: 



dtn + V • v n = 



(31) 



Combining (f30|) with (|24|) and (|28|) we finally arrive at 



0C(P e -||P c ) = 2E«) / dt\\v\\l 



(32) 



The right hand side of (|3"2"|) is proportional to the "kinetic 
energy" specified by the current velocity of the system. 
Its physical interpretation 0, H3] (see also @, 0, 41 1) 



is that of entropy production during the transformation. 
The way we arrived to (|32j) differs to some extent from 
the one of [H, 0, [H| . There the role of the Kullback- 
Leibler divergence is played by an "action functional" de- 
fined by contrasting the forward dynamics with its path- 
space time reversed defined by inverting the arrow of time 
t i-> t{ + t — t in analogy to what was done to prove 
fluctuation theorems in hyperbolic dynamics 12]. The 
entropy production relation can also be regarded as the 
continuum limit under diffusive scaling of the analogous 
relation found in 3j| for Markov jump processes. 



STOCHASTIC THERMODYNAMICS 



Let us now examine the consequences of (I32|) for op- 
timal control in thermodynamic functionals. We thus 
consider a dynamical system driven by time-dependent 
gradient-like drift 



b{x,i) 



- 1 (x)-d x U(x,t) 



(33) 



which is a stylized model of a mechanical potential sub- 
ject to an external control. The current velocity becomes 
in such a case 

v{x, t) = -g- x (x) • d x (u + i In n) (x, t) (34) 

Drawing from [42[ , we define the work done on the system 
as 



t ,t„ =rtv j\td t U(t t ,t) = E^ u(e u t)\Z 

-E«> £ (d&f g- 1 ^) • %£/(&, t)) g (35) 



If we require the first law of thermodynamics to hold 
true, we must identify the heat as 

Q t( , to = -E«> [ \d£ t <> g-Htt) -deMSt, t)) g (36) 



Under the present conventions, the local equilibrium con- 
dition is 



nd x U + 7^d x n 







(37) 



Correspondingly, we define the osmotic velocity 3^, 3 
as 

1 



u(x, t) 
1 



j (6 -&)(*,*) 
l (x) ■ d x hin(x, t) 



(38) 



and the scalar (i.e. coordinate independent) expression 
of the Gibbs-Shannon entropy 



S(i) 



Je«) Inn(€ t ,t) 

d^x (n lnn)(x,t) 



2 



(39) 



Note that the definition (13^)) of the Gibbs-Shannon en- 
tropy differs from the one given in 0, 0] by a prefactor 
1/2 dictated by the convention adopted here for the dif- 
fusion tensor, and by an addend proportional to In \/|g[ 
which does not transform as a scalar under change of 
coordinates. If we now add and subtract the osmotic 
velocity to the integrand in (13"5"|) and (|3"o]) . use probabil- 
ity conservation and the identity (1301) . we arrive at the 
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manifestly coordinate-independent representations of the 
released heat 



-[S(tf)-S(*„)]+E(« / d* \\v\\l (40) 



Qi 



and of thermodynamic work 

W tt , to = J(i f ) - J(i D ) + E«) J' At || v \\l (41) 

In (l4"T|) the Hclmholtz free energy of the system is 
J(t) = U(t) - S(i) = 

E«) (W^lnn) (&,t) :=-E«>i&(£ t ,f) (42) 

The advantage to introduce here the scalar quantity ip to 
denote the free energy density emerges when contrasting 
the definitions of the current and osmotic velocities with 
(|33[). We recognize then that the current velocity is the 
gradient of the Helmholtz free energy density: 



v(x,t) = g 1 (x) ■ d x ip(x,t) 



(43) 



It is worth noticing that the Hclmholtz free energy i/j 
density (|42|) coincides with the potential similarly de- 
noted in [J] owing to a cancellation of terms proportional 
to In \f\g\ between the individually non-coordinate- 
independent expressions of the internal energy and the 
Gibbs-Shannon entropy thereby used. Similar consider- 
ations, exploiting the time-independence of the Riemann 
metric also guarantee that the expression for the released 
heat of [IJ does coincide with the one given here. The 
same is true for the work whose expression (14"TT) has been 
obtained also in [(| using, however, arguments valid only 
in the weak noise limit. 



OPTIMAL CONTROL 

The expressions (|4T)|) and (|4"Tj) evince that the current 
velocity formulation of Langevin thermodynamics is nat- 
ural from the point of view of control theory. There 
are at least two kinds of considerations supporting this 
claim. First, by (|31[) the current velocity maps the orig- 
inal stochastic control problem into a deterministic con- 
trol one. Such a mapping exists for any smooth diffusion 
but it is of limited practical use if the drift is known a- 
priori and the problem is to derive the evolution of the 
density. It becomes useful for control purposes when the 
drift is not known but must be determined by minimiz- 
ing a "cost" functional over a suitable space of admissible 
controls. The second consideration is that the functional 
dependence of the entropy production on the current ve- 
locity readily enforces the coercivity condition (see e.g. 



11 



pag. 33: convexity with growth faster than linear) 
which plays an important role in variational calculus to 



prove the existence of extremal solutions. A direct conse- 
quence of these considerations is the recovery of Jarzyn- 
ski's 



and "finite-time Landauer's" (2|| inequalities 
Qt f ,t > -[S(tf)-S(to)] 



(44) 



(45) 



The interpretation of (|45j) as Landauer's inequality stems 
from the identification of the heat release with the en- 
tropy variation of the environment. By (|34[) the cur- 
rent velocity vanishes at equilibrium. As a consequence, 
the inequalities (|44p . (|45p become tight for transitions 
described by a jump between two "equilibrium" states. 
Physically, jumps are mathematical mock-up's for tran- 
sitions occurring at the fastest admissible time-scale. As 
such, they are not suited to describe macroscopic con- 
trol of a nano-system. It is therefore relevant to look 
for minima of thermodynamics indicators by restricting 
the space of admissible controls to those guaranteeing 
a smooth behavior of the current velocity. In order to 
achieve this goal let us recall that given a forward Markov 
process ry, deterministic or stochastic, with generator 
25 [ u ] depending on a control u, the canonical form of 
cost functional considered in control theory [ll| is 



dtL(rj t ,t;u) (46) 



Standing some regularity assumptions, it is a-priori ten- 
able to expect the "cost functional" A to admit a mini- 
mum over the space of smooth controls u if the "running 
cost" L depends coercively on u and the "terminal cost" 
iff is some assigned function independent of u. The min- 
imum 



J+(x,t) =min^l(a;,t) 



(47) 



is usually referred to as the "value function" . Under these 
hypotheses, the solution of the Hamilton- Jacobi-Bellman 
equation 



0tJ* + min{<9 [tl] J* + Z} = 



J*(a:,if) = 



(48a) 



(48b) 



corresponding to the smallest value of A (in case of mul- 
tiple minima in (j48ajl ) specifies the value of the optimal 
control it* for any t in a closed horizon [t a , tf] if the 
resulting 25 is a well-defined generator of a Markov 
process. The properties that a solution of (|48)) must en- 
joy in order to satisfy such self-consistency condition are 
determined by so called verification theorems 



ii, m 



Linearity of the Kolmogorov pair of equations govern- 
ing a Markovian dynamics extends (|4"8")> to expectations 
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of A with respect to the measure of rj evolving from 
non- localized initial density assigned at time t = t a [l5j |. 
Namely, if we denote variations with respect to the con- 
trol u by a "prime symbol" , we can couch the variation 
of E^U into the form 

(E("U)'(77 to ,i )= f d^(n; 



+ / dt / d^rr (n^L + n^L') (a:,*) (49) 
Jt D Jm 

If we now require the "dynamic programming" (non- 
homogeneous backward Kolmogorov) equation to hold 
for any admissible control 



(d t + <S [u] ) J + L = 
we obtain for n^(-,i ) = 

(eWa)'= [ d^[ n ;(*- j)]( x ,t) 



(50) 



dt / d 



{j[(9 t -0f u] K,]'}(*,t) 
+ ['dt [ d^x {n v (e' [u] J + L')\ (x,t) (51) 



The variation yields a stationary point if (|48b[) is satis- 
fied, the probability density evolves for any u according 
to the dynamics specified by the adjoint action of <§[«], 
and the optimal control is fixed by the same condition 
as in (|48al) . If, furthermore, the generator is linear in 
the control it, the convexity of L readily implies that the 
stationary point is a minimum 

(E^U)" = ^ dt f d^x (iir, L") (x, t) > (52) 

The dynamic programming equation (|50l) has the hydro- 
dynamic interpretation of a material derivative along the 
realizations of the Markov process rj. Requiring it to 
hold a priori substantiate Bellman's idea that optimal 
control stems from a_condition imposed locally during the 
time evolution 11 



l45j . Within this framework, probabil- 
ity evolution must be regarded as the adjoint transport 
equation which may-be non-local owing to the bound- 
ary conditions and must hold in the same time horizon 
[t a , tf] where (|5"0"|) is defined. In [15| it was also shown that 
the above chain of steps holds also if the running cost L 
depends upon derivatives of the control. This fact was 
originally exploited in Q to unveil the relation between 
optimal thermodynamic control and hydrodynamic mass 
transport. The formulation in terms of the current ve- 
locity renders, however, this extension no further needed 
for thermodynamic control. 

In the application of the control-theory toolkit to ex- 
pressions such as (14"TT) we are confronted with a subtle 



difficulty. While the entropy production specifies a well- 
defined coercive running cost, the interpretation of the 
terminal cost is more problematic. Namely, a candidate 
smooth optimal control must satisfy the stationarity con- 
dition 



d x J* + 2g-v = d x (J* + 2^) =0 



(53) 



and the terminal condition (we suppose that no optimiza- 
tion is carried out over the initial state) 



J*(x,if) = *(£C) 



(54) 



Contrasting (JSHJ) and (|54p with (14"3"|) it is evident that 
the terminal condition cannot be interpreted as the end- 
horizon value of the free energy density of a smooth 
optimal protocol since in general ip(-,t{) ^ ^(")- Fur- 
thermore, a process jumping at tj from a state of non- 
vanishing current velocity cannot be considered optimal, 
independently of the prescription adopted for the proba- 
bility density at time tf . Namely, if we enlarge the space 
of admissible protocols to encompass jump processes, we 
know a-priori that the infimum is attained by an instan- 
taneous transition between "equilibria". In this latter 
case the current velocity and the running cost vanish 
identically in the full control horizon and the work co- 
incides with the free energy difference. The conclusion is 
that the "optimal work" protocol proposed in [39| cannot 
be justified using Hamilton-Jacobi-Bellman theory and 
Langevin dynamics alone. It appears instead to describe 
an optimal control strategy if the Langevin dynamics is 
embedded into an higher order Markovian dynamics Q 
following ideas closely reminiscent of the "valley method" 
(see e.g. [H). The "valley method" is a technique which 
aims at justifying the stationary point approximation to 
path integrals when no exact classical field configuration 
can match the required boundary conditions. It is worth 
stating clearly that if the terminal cost (|54[) is interpreted 
only as the variation of an external potential without ex- 
plicit relation with ip in the control horizon [t , if] then 
the protocol found in 0, H^] is optimal according to stan- 
dard verification theorems (see e.g. discussion in sections 
III.5 to III.8 of [H|). 

A control problem which lends itself to a more trans- 
parent physical interpretation is that of the minimiza- 
tion of the entropy production between assigned proba- 
bility densities at the end of the control horizon. The 
analytical and numerical treatment of this problem have 
been inquired in details in Q. It is worth here to draw 
the attention some aspects of this problem not discussed 
in @, 3] • By (|52")l the problem can be equivalently for- 
mulated in terms of the minimization of the Kullback- 
Leibler divergence associated to a time reversal opera- 
tion. It is instructive to contrast thermodynamic en- 
tropy production minimization with the control prob- 
lem defining the so-called "Schrodinger diffusion" see e.g. 
0, E3, 31, |4(|. Given two probability densities at the 
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end of a control horizon and a reference diffusion pro- 
cess, the Schrodinger diffusion problem determines the 
smoothly interpolating diffusion obtained from the refer- 
ence process by a deformation of the drift under the re- 
quirement that the Kullback-Lcibler divergence between 
the two processes be at a minimum value. If the reference 
process is the Wiener process then the Schrodinger diffu- 
sion corresponds to treat as running cost instead of the 
entropy production an analogous quantity in which the 
current velocity is replaced by the forward drift of the 
process. Correspondingly, the associated control prob- 
lem is turned from deterministic to stochastic. A widely 
used approach to deterministic control is to prove exis- 
tence and uniqueness of solutions in viscosity sense [llj ]. 
In simplest cases this means constructing solutions as 
the inviscid limit of an ultraviolet regularization of the 
Hamilton- Jacobi equation by adding a Laplacian. The 
Monge- Ampere-Kantorovich method [32[ applied in Q to 
study nucleation at minimum entropy production is an 
example of this general ideology. This observation allows 
us to attribute a direct physical interpretation also to the 
regularized entropy production minimization problem. 
Namely, in the presence of any finite viscosity we can 
interpret the minimizer of the Kullback Leibler diver- 
gence (|32|) as the drift solving an associated Schrodinger 
diffusion problem. 

CONCLUSIONS 

In conclusion, we showed how a coordinate indepen- 
dent formalism for stochastic differential equations pro- 
vides a convenient formulation of control problems aris- 
ing in stochastic thermodynamics. In doing so we re- 
stricted the attention to time-independent diffusion ten- 
sors. This is not too restrictive under the hypothesis 
that mechanical forces and control parameters are most 
naturally encapsulated in the drift field whilst the dif- 
fusion coefficient contains purely geometric information. 
We also analyzed how the local nature of Bellman prin- 
ciple affects the optimal control equations. Preserving 
the adjoint structure of the Kolmogorov pair requires for 
example that jumps in the protocol which is governed by 
the backward Kolmogorov equation bring about jumps 
in the forward Kolmogorov equation governing the prob- 
ability density evolution. Hence, it is an essential mod- 
eling question to assess a-priori which is the physically 
relevant space of admissible protocols and whether and 
how to describe the procedures to switch on and off the 
optimal protocol at the end of the control horizon. 
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